fix(ai): add message truncation to openai–agents #4968

shellmayr · 2025-10-21T09:17:25Z

Add gen_ai.request.message truncation to OpenAI Agents as introduced in fix(ai): introduce message truncation for openai #4946

Closes TET-1208

linear · 2025-10-21T09:17:29Z

TET-1208 Python SDK: truncate gen_ai messages

codecov · 2025-10-24T07:45:55Z

Codecov Report

❌ Patch coverage is 80.00000% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 83.94%. Comparing base (dcd94ed) to head (5a16b2e).
⚠️ Report is 10 commits behind head on master.
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
...k/integrations/openai_agents/spans/invoke_agent.py	75.00%	0 Missing and 1 partial ⚠️
sentry_sdk/integrations/openai_agents/utils.py	83.33%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #4968      +/-   ##
==========================================
- Coverage   83.96%   83.94%   -0.03%     
==========================================
  Files         165      178      +13     
  Lines       17073    17842     +769     
  Branches     2999     3172     +173     
==========================================
+ Hits        14336    14978     +642     
- Misses       1839     1900      +61     
- Partials      898      964      +66

Files with missing lines	Coverage Δ
...k/integrations/openai_agents/spans/invoke_agent.py	`86.11% <75.00%> (-1.77%)`	⬇️
sentry_sdk/integrations/openai_agents/utils.py	`87.64% <83.33%> (-0.46%)`	⬇️

... and 20 files with indirect coverage changes

alexander-alderman-webb · 2025-10-24T10:47:50Z

tests/integrations/openai_agents/test_openai_agents.py

+    # Just verify that messages are being set and truncation is applied
+    # The actual truncation behavior is tested in the ai_monitoring tests
+    ai_client_span = ai_client_spans[0]
+    if "gen_ai.request.messages" in ai_client_span["data"]:
+        messages_data = ai_client_span["data"]["gen_ai.request.messages"]
+        assert isinstance(messages_data, str)
+
+        parsed_messages = json.loads(messages_data)
+        assert isinstance(parsed_messages, list)
+        # Verify messages were processed
+        assert len(parsed_messages) >= 1


Is this checking that truncation is applied?

I would have thought len(parsed_messages) >= 1 is always true, even without truncation.

Always use "hf-inference" provider for chat completions, which is the existing behavior of our tests prior to version 1.0.0rc7 of `huggingface_hub`. Closes #4978

#### Issues * resolves: #4975 * resolves: PY-1908

### Description Added 3.14 support to package meta and reran toxgen so that it detects we support 3.14 now and updates the test matrix. #### Issues - Closes #4911 - Closes #4910 #### Reminders - Please add tests to validate your changes, and lint your code using `tox -e linters`. - Add GH Issue ID _&_ Linear ID (if applicable) - PR title should use [conventional commit](https://develop.sentry.dev/engineering-practices/commit-messages/#type) style (`feat:`, `fix:`, `ref:`, `meta:`) - For external contributors: [CONTRIBUTING.md](https://github.com/getsentry/sentry-python/blob/master/CONTRIBUTING.md), [Sentry SDK development docs](https://develop.sentry.dev/sdk/), [Discord community](https://discord.gg/Ww9hbqr)

Update `split_tox_gh_actions.py` to support the free-threading `t` suffix in versions. Upgrade `pip` in tox for free-threaded Python by setting `VIRTUALENV_PIP`. Check sys.flags.thread_inherit_context to determine if context is already propagated when creating threads in the `threading` tests. See https://docs.python.org/3/howto/free-threading-python.html#context-variables Closes #4970

### Description Adapt to import changes in new langchain 1.x #### Issues Closes #4735 #### Reminders - Please add tests to validate your changes, and lint your code using `tox -e linters`. - Add GH Issue ID _&_ Linear ID (if applicable) - PR title should use [conventional commit](https://develop.sentry.dev/engineering-practices/commit-messages/#type) style (`feat:`, `fix:`, `ref:`, `meta:`) - For external contributors: [CONTRIBUTING.md](https://github.com/getsentry/sentry-python/blob/master/CONTRIBUTING.md), [Sentry SDK development docs](https://develop.sentry.dev/sdk/), [Discord community](https://discord.gg/Ww9hbqr)

### Description Integration for Pydantic AI. Support for sync and async (streaming and non-streaming) #### Issues Closes https://linear.app/getsentry/issue/TET-1242/python-sdk-pydantic-ai-integration #### Reminders - Please add tests to validate your changes, and lint your code using `tox -e linters`. - Add GH Issue ID _&_ Linear ID (if applicable) - PR title should use [conventional commit](https://develop.sentry.dev/engineering-practices/commit-messages/#type) style (`feat:`, `fix:`, `ref:`, `meta:`) - For external contributors: [CONTRIBUTING.md](https://github.com/getsentry/sentry-python/blob/master/CONTRIBUTING.md), [Sentry SDK development docs](https://develop.sentry.dev/sdk/), [Discord community](https://discord.gg/Ww9hbqr) --------- Co-authored-by: Ivana Kellyer <ivana.kellyer@sentry.io>

### Description Adds integration for the [MCP Python SDK](https://github.com/modelcontextprotocol/python-sdk) #### Issues * Closes https://linear.app/getsentry/issue/TET-1267/mcp-python-sdk-integration * Closes https://linear.app/getsentry/issue/TET-1301/python-mcp-add-tests --------- Co-authored-by: Ivana Kellyer <ivana.kellyer@sentry.io>

### Description 1. Remove `potel-base` from GH action triggers. 2. Let's use the name `major/<version>` for major branches from now on. Added the pattern to our test workflow files so that we don't need to add each new major branch to them by hand in the future. 3. Pulled in new test matrix updates as a side effect of regenerating the YAMLs. #### Issues Closes #5004 #### Reminders - Please add tests to validate your changes, and lint your code using `tox -e linters`. - Add GH Issue ID _&_ Linear ID (if applicable) - PR title should use [conventional commit](https://develop.sentry.dev/engineering-practices/commit-messages/#type) style (`feat:`, `fix:`, `ref:`, `meta:`) - For external contributors: [CONTRIBUTING.md](https://github.com/getsentry/sentry-python/blob/master/CONTRIBUTING.md), [Sentry SDK development docs](https://develop.sentry.dev/sdk/), [Discord community](https://discord.gg/Ww9hbqr)

Update our test matrix with new releases of integrated frameworks and libraries. ## How it works - Scan PyPI for all supported releases of all frameworks we have a dedicated test suite for. - Pick a representative sample of releases to run our test suite against. We always test the latest and oldest supported version. - Update [tox.ini](https://github.com/getsentry/sentry-python/blob/master/tox.ini) with the new releases. ## Action required - If CI passes on this PR, it's safe to approve and merge. It means our integrations can handle new versions of frameworks that got pulled in. - If CI doesn't pass on this PR, this points to an incompatibility of either our integration or our test setup with a new version of a framework. - Check what the failures look like and either fix them, or update the [test config](https://github.com/getsentry/sentry-python/blob/master/scripts/populate_tox/config.py) and rerun [scripts/generate-test-files.sh](https://github.com/getsentry/sentry-python/blob/master/scripts/generate-test-files.sh). See [scripts/populate_tox/README.md](https://github.com/getsentry/sentry-python/blob/master/scripts/populate_tox/README.md) for what configuration options are available. _____________________ _🤖 This PR was automatically created using [a GitHub action](https://github.com/getsentry/sentry-python/blob/master/.github/workflows/update-tox.yml)._ --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Ivana Kellyer <ivana.kellyer@sentry.io>

### Description #### Problem There are two ways to instrument strawberry-graphl apps: via a sync or an async extension. We enable one or the other based on what web framework is installed (async framework -> async extension, sync framework -> sync extension). This auto-detection can be overridden via an integration option. At some point (SDK 2.0?), we added `StrawberryIntegration` to auto-enabling integrations, which means the brittle auto-detection kicks in as soon as someone has `strawberry-graphl` installed. This can lead to issues, most notably when we auto-enable the async version of the extension even though the user's Strawberry app is actually sync. #### Options 1. Removing the auto-detection, always enabling the sync version if not specified otherwise. This way we'll never mistakenly enable async code in a sync app. We also had a [report](#3670 (comment)) at some point that the sync extension actually performs better than async, so enabling sync by default shouldn't be a big problem in async apps. 2. Removing `StrawberryIntegration` from auto-enabling integrations. Breaking change. People might just lose their traces and errors. 3. Improving the auto-detection. Best option, but out of ideas how to do this. Went with 1), all things considered it's the least breaking change (unless there's a way to do 3). Needs a big callout in the changelog anyway though. #### Issues Closes #4980 #### Reminders - Please add tests to validate your changes, and lint your code using `tox -e linters`. - Add GH Issue ID _&_ Linear ID (if applicable) - PR title should use [conventional commit](https://develop.sentry.dev/engineering-practices/commit-messages/#type) style (`feat:`, `fix:`, `ref:`, `meta:`) - For external contributors: [CONTRIBUTING.md](https://github.com/getsentry/sentry-python/blob/master/CONTRIBUTING.md), [Sentry SDK development docs](https://develop.sentry.dev/sdk/), [Discord community](https://discord.gg/Ww9hbqr)

We've been noticing some discards from clients for likely buffer reason. Metrics were initially set to 100 to match logs but the size limit for a metric in relay is ~1000x less (~1-2kb at the moment) so the memory pressure should be fine (it'd cap in the single digit megabyte range).

Moves metrics code from `_metrics.py` to `metrics.py`, and updates metrics tests to account for the namespace change.

Determine cache hits and misses independently for `django.core.cache.get()` and `django.core.cache.get_many()`. When calling `get_many()`, only register a cache miss when the returned value is an empty dictionary. When calling `get()`, register a cache miss when - the return value is `None` and no default value is provided; or - the return value is equal to a provided default value. Closes #5027

… an execute tool span should be set to error (#4986)

### Description Cherry-picked off of #5030 #### Issues Ref https://linear.app/getsentry/issue/TET-1293/make-sure-that-agent-name-is-set-on-all-of-its-gen-ai-children #### Reminders - Please add tests to validate your changes, and lint your code using `tox -e linters`. - Add GH Issue ID _&_ Linear ID (if applicable) - PR title should use [conventional commit](https://develop.sentry.dev/engineering-practices/commit-messages/#type) style (`feat:`, `fix:`, `ref:`, `meta:`) - For external contributors: [CONTRIBUTING.md](https://github.com/getsentry/sentry-python/blob/master/CONTRIBUTING.md), [Sentry SDK development docs](https://develop.sentry.dev/sdk/), [Discord community](https://discord.gg/Ww9hbqr) --------- Co-authored-by: Fabian Schindler <fabian.schindler@sentry.io>

Forgot to call this out in the changelog. Fixing that.

Modifies `populate_tox.py` to add free-threading Python to the test suite for integrations where we can expect the package to support free-threading. Packages are considered compatible with free-threading if the package itself, and all its dependencies, support free-threading. Pure Python packages are considered compatible if their dependencies support free-threading. The support for free-threading of packages with extension modules is determined based on the ABI tag in the wheel name. A dry-run of `pip install` determines the ABI tags of dependencies. Relevant output from the command is cached analogously to responses of PyPI REST calls. Closes #4971

shellmayr · 2025-10-30T14:27:30Z

Gah, screwed this one up with a bad git command; apologies, will open a new one

cursor · 2025-10-30T14:28:43Z

tests/integrations/openai_agents/test_openai_agents.py

+    assert isinstance(parsed_messages, list)
+    assert len(parsed_messages) == 2
+    assert "small message 4" in str(parsed_messages[0])
+    assert "small message 5" in str(parsed_messages[1])


Bug: Test Fails to Validate Message Truncation

The test_openai_agents_message_truncation test uses assert len(parsed_messages) >= 1, which doesn't confirm that message truncation actually occurred. This assertion passes even if truncation is broken, making the test ineffective at validating the feature.

Additional Locations (2)

tests/integrations/openai_agents/test_openai_agents.py#L1158-L1159

tests/integrations/google_genai/test_google_genai.py#L945-L947

cursor · 2025-10-30T14:28:43Z

tests/integrations/openai_agents/test_openai_agents.py

+        assert isinstance(parsed_messages, list)
+        # Verify messages were processed
+        assert len(parsed_messages) >= 1
+


Bug: Test Fails to Validate Message Truncation

The assertion in the async test_openai_agents_message_truncation doesn't properly verify message truncation. This allows the test to pass even if truncation fails, making it ineffective at validating the intended functionality.

fix(ai): add message truncation to openai–agents

629cbb1

normalize data in messages

5a16b2e

shellmayr marked this pull request as ready for review October 24, 2025 07:32

shellmayr requested a review from a team as a code owner October 24, 2025 07:32

This comment was marked as outdated.

Sign in to view

alexander-alderman-webb reviewed Oct 24, 2025

View reviewed changes

alexander-alderman-webb and others added 22 commits October 30, 2025 15:18

tests(huggingface): Support 1.0.0rc7 (#4979)

0afac42

Always use "hf-inference" provider for chat completions, which is the existing behavior of our tests prior to version 1.0.0rc7 of `huggingface_hub`. Closes #4978

Make logger template format safer to missing kwargs (#4981)

75d0cfe

#### Issues * resolves: #4975 * resolves: PY-1908

fix(ai): add message truncation to litellm (#4973)

7cfd63b

fix(ai): truncate messages for google genai (#4992)

eaa8d42

fix startlette deprecation warning (#5034)

85dbd04

chore(metrics): Rename _metrics to metrics (#5035)

24b8f15

Moves metrics code from `_metrics.py` to `metrics.py`, and updates metrics tests to account for the namespace change.

fix(integrations): hooking into error tracing function to find out if…

4e6b9d9

… an execute tool span should be set to error (#4986)

release: 2.43.0

2320823

Update CHANGELOG.md

f3b57b4

docs: Elaborate on Strawberry autodetection in changelog (#5039)

2dfa9c1

Forgot to call this out in the changelog. Fixing that.

alexander-alderman-webb and others added 3 commits October 30, 2025 15:23

fix(ai): add message truncation to openai–agents

2409cb3

make tests useful

9815f2b

shellmayr closed this Oct 30, 2025

cursor bot reviewed Oct 30, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix(ai): add message truncation to openai–agents #4968

fix(ai): add message truncation to openai–agents #4968

Uh oh!

shellmayr commented Oct 21, 2025 •

edited

Loading

Uh oh!

linear bot commented Oct 21, 2025

Uh oh!

This comment was marked as outdated.

Uh oh!

codecov bot commented Oct 24, 2025 •

edited

Loading

Uh oh!

alexander-alderman-webb Oct 24, 2025 •

edited

Loading

Uh oh!

shellmayr commented Oct 30, 2025

Uh oh!

cursor bot Oct 30, 2025

Uh oh!

cursor bot Oct 30, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

fix(ai): add message truncation to openai–agents #4968

fix(ai): add message truncation to openai–agents #4968

Uh oh!

Conversation

shellmayr commented Oct 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

linear bot commented Oct 21, 2025

Uh oh!

This comment was marked as outdated.

Uh oh!

codecov bot commented Oct 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

alexander-alderman-webb Oct 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

shellmayr commented Oct 30, 2025

Uh oh!

cursor bot Oct 30, 2025

Choose a reason for hiding this comment

Bug: Test Fails to Validate Message Truncation

Uh oh!

cursor bot Oct 30, 2025

Choose a reason for hiding this comment

Bug: Test Fails to Validate Message Truncation

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

shellmayr commented Oct 21, 2025 •

edited

Loading

codecov bot commented Oct 24, 2025 •

edited

Loading

alexander-alderman-webb Oct 24, 2025 •

edited

Loading